Tweets as impact indicators: Examining the implications of automated bot accounts on Twitter

This brief communication presents preliminary findings on automated Twitter accounts distributing links to scientific papers deposited on the preprint repository arXiv. It discusses the implication of the presence of such bots from the perspective of social media metrics (altmetrics), where mentions of scholarly documents on Twitter have been suggested as a means of measuring impact that is both broader and timelier than citations. We present preliminary findings that automated Twitter accounts create a considerable amount of tweets to scientific papers and that they behave differently than common social bots, which has critical implications for the use of raw tweet counts in research evaluation and assessment. We discuss some definitions of Twitter cyborgs and bots in scholarly communication and propose differentiating between different levels of engagement from tweeting only bibliographic information to discussing or commenting on the content of a paper. Introduction The so-called altmetrics movement promotes the use of metrics generated from social media platforms as broader and timelier evidence of research impact than citations (Priem, Taraborelli, Groth, & Neylon, 2010). Among these social media platforms, the microblogging platform Twitter has been shown to be one of the most frequently used social media tools that mentions scholarly documents (Costas, Zahedi, & Wouters, 2014; Haustein et al., 2014b; 2014c). As of July 2014, Twitter reported 500 million tweets per day and 255 million monthly active users with 16% of the U.S. population purportedly on Twitter. Scholarly use is proportionally less: it is estimated that between 3% and 15% of researchers utilize this service (Priem & Costello, 2010; Pscheida, Albrecht, Herbst, Minet, & Köhler, 2013; Rowlands, Nicholas, Russell, Canty, & Watkinson, 2011). Therefore, some authors have argued that Twitter is potentially able to capture the impact of scholarly documents on a nonresearching public and might, thus, serve as a measure of broader impact of research. However, systematic evidence as to whether tweets are valid markers of actual societal and/or scientific impact